Does AHP help us make a choice? An experimental evaluation

In this paper, we use experimental economics methods to test how well Analytic Hierarchy Process (AHP) fares as a choice support system in a real decision problem. AHP provides a ranking that we statistically compare with three additional rankings given by the subjects in the experiment: one at the beginning, one after providing AHP with the necessary pair-wise comparisons and one after learning the ranking provided by AHP. While the rankings vary widely across subjects, we observe that for each individual all four rankings are similar. Hence, subjects are consistent and AHP is, for the most part, able to replicate their rankings. Furthermore, while the rankings are similar, we do find that the AHP ranking helps the decision-makers reformulate their choices by taking into account suggestions made by AHP. Subject areas: Decision analysis, Multiple Criteria Decision Aid, Analytic Hierarchy Process (AHP), Validation, Experimental Economics